n  the  May-June  2013  issue  of  Defense  AT&L  magazine,  Under  Secretary  of  Defense  for  Acqui¬ 
sition,  Technology  and  Logistics  Frank  Kendall  authored  an  article  titled  "The  Original  Better 
Buying  Power— David  Packard  Acquisition  Rules  1971."  Packard's  fifth  acquisition  rule  was 
"fly  before  you  buy”— the  underpinnings  of  test  and  evaluation  (T&E).  I  thought  it  might  be 
interesting  to  look  at  another  challenge  that  then  Deputy  Secretary  of  Defense  David  Pack¬ 
ard  confronted  in  1971:  what  to  do  about  test  and  evaluation  in  the  Department  of  Defense 
(DoD).  The  original  Director  for  Test  and  Evaluation  created  by  David  Packard  was  a  DT&E 
with  broad  responsibilities  for  all  T&E  matters  in  the  DoD.  Today's  DT&E  is  focused  only  on 
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Developmental  Test  and  Evaluation.  This  article  provides  a 
brief  snapshot  into  the  challenges  involved  in  forming  the  first 
DT&E  office  in  the  Pentagon  and  its  evolution  into  the  DT&E 
office  today. 

In  July  1970,  President  Nixon's  Blue  Ribbon  Defense  Panel 
(BRDP)  reported  its  findings.  The  "Fitzhugh  Commission,"  as 
it  is  sometimes  called,  in  recognition  of  its  chairman,  Gilbert 
W.  Fitzhugh,  had  been  appointed  1  year  earlier  with  the  broad 
charter  to  report  and  make  recommendations  on  the  organiza¬ 
tion  and  management  of  the  DoD,  its  research  and  develop¬ 
ment  efforts,  and  its  procurement  policies  and  practices.  The 
final  report  prominently  featured  a  discussion  of  operational 
test  and  evaluation,  and  the  panel's  recommendations  had 
significant  influence  on  how  David  Packard  would  shape  the 
role  of  T&E  to  support  defense  acquisition.  (The  BRDP  is  avail¬ 
able  at  http://www.dtic.mil/dtic/tr/fulltext/u2/a013261.pdf .) 


responsibilities  for  any  evaluation  function  must  be  exercised 
independently.  When  they  are  subordinated  to  or  combined 
with  responsibilities  for  the  development  of  the  item  or  subject 
being  evaluated,  the  requisite  objectivity  is  seriously  jeopar¬ 
dized."  The  BRDP  also  considered  evaluation  within  a  much 
broader  construct  and  recommended  that  the  Secretary  create 
a  position  of  "Deputy  Secretary  of  Defense  for  Evaluation"  sup¬ 
ported  by  three  Assistant  Secretaries:  Comptroller,  Program 
and  Force  Analysis,  and  Test  and  Evaluation.  The  BRDP  fur¬ 
ther  recommended  that  the  Secretary  create  a  Defense  Test 
Agency  "to  perform  the  functions  of  overview  of  all  Defense 
test  and  evaluation,  designing  or  reviewing  of  designs  for  test, 
monitoring  and  evaluation  of  the  entire  Defense  test  program, 
and  conducting  tests  and  evaluations  as  required,  with  par¬ 
ticular  emphasis  on  operational  testing,  and  on  systems  and 
equipments  which  span  Service  lines." 


The  BRDP  had  three  main  findings  relative  to  the  conduct  of 
T&E:  "functional  testing"  (we  use  the  term  "developmental 
testing"  today)  is  fundamentally  sound;  operational  testing  is 
generally  inadequate,  and  the  Services  do  not  conduct  enough 
joint  test  and  evaluation.  Regarding  "functional"  testing,  the 
BRDP  wrote: 

By  and  large,  functional  testing  in  and  for  the  Department  of 
Defense  appears  to  be  well  understood  and  faithfully  executed. 
Serious  policy  deficiencies  are  not  apparent,  and  such  failures 
in  functional  testing  as  occur  can  be  primarily  attributed  to  lack 
of  technical  competence,  oversight,  or  procedural  breakdowns. 
Functional  testing  is  not  considered  to  be  a  major  problem  area. 

Pertaining  to  operational  test  and  evaluation,  however,  the 
BRDP  found  that  "Operational  test  and  evaluation  has  been 
too  infrequent,  poorly  designed  and  executed,  and  generally 


Packard  promptly  began  acting  on  the  BRDP  recommenda¬ 
tions.  In  the  first  8  months  of  1971,  he  signed  three  memoranda 
that  made  sweeping  changes  to  the  role  of  test  and  evaluation 
in  support  of  defense  acquisition. 

In  the  first  memorandum,  "Conduct  of  Operational  Test  and 
Evaluation,"  dated  Feb.  11, 1971,  Packard  wrote, "...  a  number 
of  specific  actions  must  be  taken  to  put  our  house  in  order  in 
this  very  important  aspect  of  the  weapon  system  acquisition 
process."  Packard  directed  three  specific  actions: 

■  Services  will  restructure  their  organizations  for  OT&E  to  be 
"separate  and  distinct  from  the  developing  command"  and 
report  the  results  of  its  test  and  evaluation  efforts  "directly 
to  the  Chief  of  the  Service." 

■  OSD  will  establish  "a  Deputy  Director  for  Test  and 
Evaluation  within  the  Office  of  Director  of  Defense  Re¬ 
search  and  Engineering  (ODDR&E)  with  across-the-board 


inadequate."  The  panel's  recommendations  regarding  T&E, 
and  thus  the  actions  taken  by  DoD  to  implement  those  recom¬ 
mendations,  essentially  focused  on  correcting  deficiencies  in 
operational  test  and  evaluation  (OT&E),  and  have  driven  the 
primary  focus  in  defense  acquisition  for  the  past  40  years. 

The  BRDP  also  highlighted  the  lack  of  OT&E  oversight  in 
OSD  as  a  "glaring  deficiency"  and  reported  that  "In  connec¬ 
tion  with  test  and  evaluation,  it  should  be  emphasized  that 


responsibilities  for  OSD  in  test  and  evaluation  matters.  This 
office  will  review  and  approve  test  and  evaluation  plans 
prepared  by  the  Services  and  will  provide  an  assessment 
of  results  obtained." 

■  Services  will  conduct  more  joint  operational  test  and 
evaluation. 

In  the  Feb.  11, 1971,  memo,  Packard  asked  the  Service  Secre¬ 
taries  to  advise  him  of  their  plans  before  March  31,  and  to  "be 
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prepared  to  have  the  new  structure  in  effect  by  the  end  of  this 
fiscal  year.”  Prior  to  1976,  the  fiscal  year  began  on  July  1  and 
ended  on  June  30;  in  other  words,  Packard  gave  the  DoD  just 
4  months  to  put  its  house  in  order!  Clearly,  this  was  a  high 
priority  for  the  Deputy  Secretary. 


Evaluation  would  be  responsible  for  “initiating  and  coordinat¬ 
ing  appropriate  joint  testing;  overseeing  the  test  and  evaluation 
of  foreign  systems  for  possible  DoD  use;  and  administering 
for  OSD  its  responsibilities  for  the  national  and  major  Service 
test  ranges.” 


The  second  of  Packard's  memoranda,  dated  April  21, 1971,  ce¬ 
mented  the  role  of  the  Deputy  Director  for  T&E  in  the  Defense 
System  Acquisition  Review  Council  (DSARC)  (the  DSARC  was 
the  forerunner  of  the  modern  Defense  Acquisition  Board). 


Packard  needed  someone  to  lead  the  new  organization  and 
make  his  vision  reality.  On  June  7, 1971,  Packard  appointed 
retired  Lt.  Gen.  Alfred  Dodd  Starbird  to  be  the  first  Deputy 
Director  for  Test  and  Evaluation.  Lt.  Gen.  Starbird  set  out  to 


Packard  directed  the  Military  Departments  to  provide  their 
assessment  of  test  results  to  the  DDR&E  prior  to  the  DSARC 
Milestone  III  production  decision.  The  Deputy  Director  for 
Test  and  Evaluation  (DDT&E)  would  review  those  results  and 
provide  his  independent  recommendations  at  the  Milestone 
III  meeting. 

Packard  also  quickly  began  honing  the  new  processes  as  they 
were  implemented.  On  Aug.  3, 1971,  Packard  wrote,  “Based  on 
experience  to  date  resulting  from  [the  previous  two  memos], 
I  consider  certain  further  clarification  and  instruction  is  re¬ 
quired.”  In  this  third  memorandum,  “Test  and  Evaluation  in 
System  Acquisition  Process,”  Packard  described  in  more  de¬ 
tail  requirements  for  conduct  of  adequate  OT&E.  Addition¬ 
ally,  Packard  assigned  these  additional  responsibilities  to  the 
Deputy  Director  for  Test  and  Evaluation: 

■  “be  responsible  within  OSD  for  reviewing  test  and  evaluation 
policies  and  procedures  and  recommendingto  me  changes 
as  appropriate; 

■  “monitor  closely  test  and  evaluation  conducted  by  the  Ser¬ 
vices  for  DSARC  programs,  and  such  other  programs  as  he 
believes  necessary,  throughout  the  entire  testing  cycle; 

■  “report  to  the  DSARC  and  directly  to  me  at  DSARC  Mile¬ 
stones!  and  II  his  assessment  as  to  the  adequacy  of  the  list 
of  critical  issues  and  problems  to  be  attacked  by  test  and 
evaluation  and  the  schedule  of  test  milestones; 

■  and  “report  at  Milestone  III  to  the  DSARC  and  directly  to  me 
his  independent  recommendation.” 

The  Deputy  Secretary  went  on  to  require  that  “Service  test 
plans  and  test  results  be  made  available  at  his  request  as  early 
as  developed.”  Additionally,  the  Deputy  Director  for  Test  and 


build  the  new  DDT&E  organization.  He  had  three  assistant 
directors  with  responsibilities  for  Tactical  Systems  T&E,  Stra¬ 
tegic  and  Support  Systems  T&E,  and  Test  Resources.  During 
his  tenure,  Starbird  recommended  the  Defense  Science  Board 
(DSB)  examine  test  and  evaluation  in  the  department.  The  DSB 
published  the  first  report  on  T&E  in  April  1974;  it  would  be  the 
first  of  six  reports  the  DSB  would  issue  on  T&E  over  the  next 
40  years.  Starbird  also  initiated  the  publication  of  the  first  DoD 
Directive  on  test  and  evaluation:  DoD  Directive  5000.3,  Test 
and  Evaluation,  was  published  on  Jan.  19, 1973.  The  5000.3 
continued  in  effect,  with  four  updates,  until  1991,  when  the 
February  1991  issuance  of  DoD  Directive  5000.1  canceled  the 
5000.3  and  T&E  became  Part  8  of  DoD  Instruction  5000.2. 
Starbird  also  was  able  to  insert  into  the  defense  budget  a  new 
appropriation  for  the  deputy  director  of  T&E.  The  Department 
of  Defense  Appropriation  Act  for  FY 1973,  Public  Law  92-570, 
dated  Oct.  26, 1972,  included  the  following  language: 

DIRECTOR  OF  TEST  AND  EVALUATION,  DEFENSE 

For  expenses,  not  otherwise  provided  for,  of  independent  ac¬ 
tivities  of  the  Director  of  Defense  Test  and  Evaluation  in  the 
direction  and  supervision  of  test  and  evaluation,  including  initial 
operational  testing  and  evaluation;  and  performance  of  joint 
testing  and  evaluation;  and  administrative  expenses  in  connec¬ 
tion  therewith,  $27,000,000,  to  remain  available  for  obligation 
until  June  30,1974. 

Thus  did  David  Packard  implement  the  recommendations  of 
the  BRDP  Although  there  would  not  be  a  Deputy  Secretary  for 
Evaluation,  or  an  Assistant  Secretary  for  T&E,  he  did  create  the 
“defense  test  agency”  in  the  form  of  the  Office  of  the  Deputy 
Director  for  Test  and  Evaluation  in  the  Office  of  the  Director 
of  Defense  Research  and  Engineering.  The  Deputy  Director 
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for  T&E  was  responsible  for  all  matters  involving  T&E  almost 
exactly  as  the  BRDP  recommended.  Importantly,  Packard  also 
ensured  the  Deputy  Director  for  T&E  provided  his  independent 
assessment  at  the  DSARC.  However,  independence  came  to 
be  another  issue  entirely,  and  the  T&E  organization  Packard 
put  in  place  would  not  endure. 

Concerns  about  OT&E  continued  to  consume  attention  in  the 
Pentagon  and  especially  on  the  Hill.  Likewise,  there  was  an 
overwhelming  sense  that  assignment  of  the  Deputy  Director 
for  Test  and  Evaluation  under  the  DDR&E  posed  a  conflict  of 
interest.  The  DDR&E  was  the  DoD's  chief  acquisition  official 
at  the  time,  which  violated  the  BRDP's  premise  that  when  re¬ 
sponsibilities  for  evaluation  "are  subordinated  to  or  combined 
with  responsibilities  for  the  development  of  the  item  or  subject 
being  evaluated,  the  requisite  objectivity  is  seriously  jeopar¬ 
dized."  I  doubt  that  Starbird,  or  any  of  the  deputies  after  him, 
felt  that  way,  but  regardless,  in  September  1983,  Congress 
established  a  presidential^  appointed,  Senate-confirmed  posi¬ 
tion  of  Director,  Operational  Test  and  Evaluation,  independent 
of  the  acquisition  authority  and  reporting  directly  to  the  Secre¬ 
tary  of  Defense.  The  position  of  Deputy  Director  for  Test  and 
Evaluation  was  renamed  Deputy  Director  for  Developmental 
Test  and  Evaluation.  Another  16  years  later,  on  June  7, 1999— 
or  28  years  after  Packard  created  it— Secretary  of  Defense 
William  Cohen  disestablished  the  test  office  within  what  had 
become  the  Office  of  the  Undersecretary  of  Defense  for  Ac¬ 
quisition  and  Technology.  During  those  nearly  3  decades,  all 


emphasis  in  T&E  in  the  department  continued  to  be  on  opera¬ 
tional  testing,  and  Cohen's  decision  was  intended  specifically 
to  strengthen  the  Office  of  the  DOT&E;  however,  it  virtually 
eliminated  oversight  of  developmental  test  and  evaluation. 
Congress  would  reverse  this  10  years  later.  How  that  came  to 
pass  is  another  story. 

As  a  member  of  the  DT&E  organization  in  OSD  today,  my  view 
of  test  and  evaluation  is  obviously  biased.  But  with  15  years' 
experience  in  the  T&E  community,  most  of  which  has  involved 
operational  testing  in  some  capacity,  I  have  come  to  believe 
that  the  department's  decades-long  emphasis  on  OT&E  in  the 
acquisition  life  cycle  took  our  eye  off  the  target.  Acquisition 
success  is  not  about  passing  OT&E  and  getting  that  full-rate 
production  decision;  success  occurs  when  we  have  properly 
set  the  conditions  to  begin  production.  The  key  to  improv¬ 
ing  acquisition  outcomes  is  to  get  the  development  right,  and 
verify  it  through  rigorous  developmental  test  and  evaluation. 
We  have  to  do  a  better  job  helping  programs  get  to  Milestone 
C;  that's  the  target  we  must  aim  for.  In  T&E,  we  have  to  shift 
left  if  we  are  going  to  "fly  before  you  buy."  Packard  had  a  good 
plan  back  in  1971,  but  we  lost  track  of  the  target.  Given  his  un¬ 
precedented  (and  unmatched)  level  of  involvement  in  shaping 
the  course  for  T&E  in  the  department,  I  think  one  could  make 
a  convincing  argument  that  David  Packard  was,  himself,  the 
original  DT&E.  & 


The  author  can  be  contacted  at  steven.j.hutchison.civ@mail.mil. 
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